Multimedia information is a combination of different types of information (e.g., still images, text, graphics, audio, and motion video) integrated into a single presentation. The growing market for multimedia presentations has stimulated interest in efficient and cost-effective mechanisms for delivering multimedia presentations from providers to customers.
Some prior delivery schemes provide real-time delivery of multimedia presentations from an information warehouse, where multimedia presentations of one or more providers are stored, to multiple customers. Illustratively, FIG. 1A depicts a prior delivery system 10 which includes an information warehouse 20, a distribution network 30, and a plurality of customer premises equipments (CPEs) 40-1 through 40-K corresponding to customers 41-1 through 41-K, respectively. The information warehouse 20 includes mass storage 21, where multimedia presentations are stored, and a server 22, which retrieves multimedia presentations from storage 21 and delivers these presentations to the CPEs of requesting customers.
The multimedia presentation includes objects which contain the images, text, graphics, or other information to be presented to the customer. On receiving a customer's request for a multimedia presentation, the server sends to the customer's CPE, a script corresponding to the multimedia presentation, which script contains instructions for retrieving the objects from the server and presenting the objects to the customer. The CPE then plays out the presentation to the customer as instructed in the script along a main presentation path and sends requests for objects of the presentation to the server 22 as needed. On receiving a request from a customer for a multimedia object, the server delivers the object to the customer's CPE as requested as quickly as possible, thereby operating in a reactive mode in delivering objects to the CPE. This reactive mode of operation between the server and the CPE characterizes the conventional client-server model.
At times during the presentation, the customer may have the opportunity to select interactive features. Such features may include moving to another screen of information or selecting the display of a sidebar. Should the customer select an interactive feature, the customer's CPE sends a request to the server for the appropriate objects of the presentation to be played out in response to the customer's selection. The server reacts by sending the requested objects of the presentation as quickly as possible to the CPE. The CPE switches from presenting the multimedia presentation along the main presentation path to presenting the multimedia presentation along a sub-presentation path corresponding to the selected interactive feature.
For purposes of illustrating the shortcomings of a reactive mode of operation characterizing prior multimedia delivery systems, FIGS. 1B and 1C depict a table and graph, respectively, of objects representing an original multimedia presentation (OMP). An OMP is used to clearly specify the behavior of a multimedia presentation. In FIG. 1B, each row corresponds to an object and the column entries represent the following:
Object Number: a unique identifier for each object.
Object Description: a comment (meant for the reader) that describes the object's purpose. The Object Description is for illustrative purposes only and would not be part of an OMP in practice.
Object Type: the medium contained in the object.
Object Size: the size of the object in bytes.
Presentation Times: the time at which the presentation of the object is to begin (Start) and the time at which the presentation of the object is to end (End). For visible images, the End Time is the time at which the object will be removed from the display. For audio objects, the End Time is the time at which the audio will no longer be audible to the customer.
Clock: the logical clock used to base this object's Start and End Times. Logical clocks are used to represent alternative paths through the presentation (paths being chosen as a result of an interactive feature). By convention, the presentation begins on Clock 1. In this example, Clock 2 represents the only alternative sub-presentation path in this presentation.
The OMP of FIGS. 1B and 1C is simplified and may not be exemplary of typical multimedia presentations. However, this illustrative presentation serves our purposes of presenting an easy-to-understand example. As shown in FIGS. 1B and 1C, our illustrative multimedia presentation comprises a main presentation which includes Objects 1 through 5 and a sub-presentation which includes Objects 6 and 7. The objects of the main presentation have presentation times based on Clock 1, and the objects of the sub-presentation have presentation times based on Clock 2.
As shown in FIGS. 1B and 1C, the OMP begins along the main presentation path and the active presentation clock is Clock 1. After a 5-second delay, the image of Object 1 is presented at t.sub.1 =5 seconds, and this image remains visible until t.sub.1 =20 seconds. Object 2, a sidebar icon, appears 9 seconds into the main presentation indicating to the customer that there is additional, optional information that may be viewed by choosing the sidebar interactive feature. The icon of Object 2 remains for 10 seconds until t.sub.1 =19 seconds. FIG. 1D depicts an interaction table for the OMP which shows that the interactive feature is enabled during the 10 second interval from t.sub.1 =9 seconds to t.sub.1 =19 seconds, and disabled at all other times. Each row of the interaction table corresponds to an interactive feature, and the meaning of the column entries in FIG. 1D are as follows:
Enable Time: the time at which this interactive option is enabled (the interactive feature can not be chosen unless it is enabled).
Disable Interaction Time: the time at which the interaction is disabled.
Interaction Clock: the Clock on which to base the Enable and Disable Times of the interactive features.
Jump Event Description: a description of the user interaction that will result in this event being selected. In practice, this entry would hold one of a number of possibilities, such as "user selects object on screen" or "user types in a number".
Clock: the clock on which the presentation will continue.
If the customer selects the interactive feature during this interval, the customer's CPE jumps from requesting and presenting objects along the main presentation path to the sub-presentation path. The screen is cleared of the main presentation objects, the active clock is now Clock 2 (t.sub.2 =0) corresponding to the sub-presentation path, and the image of Object 6 appears at t.sub.2 =0 remaining visible for 10 seconds. After t.sub.2 =10 seconds, the image of Object 7 appears and remains visible for 10 seconds until t.sub.2 =20 seconds. The sub-presentation is terminated at t.sub.2 =20 seconds, and presentation along the main presentation path is then resumed at the point in time where interrupted by the sub-presentation.
If the customer does not select the sidebar interactive feature during the 10 second interval, the presentation continues along the main presentation path without interruption. At 16 seconds into the main presentation (t.sub.1 =16 seconds), the image of Object 3 appears alongside the previous image of Object 1 and the sidebar icon of Object 2 and remains until t.sub.1 =25 seconds. At 17 seconds into the main presentation (t.sub.1 =17 seconds), the image of Object 4 similarly appears, alongside the previous images and remains until t.sub.1 =25 seconds. At 19 seconds into the main presentation (t.sub.1 =19 seconds), the sidebar icon disappears, the interactive feature is disabled, and selecting the sidebar has no effect. At t.sub.1 =25 seconds, the previous images of Objects 3 and 4 disappear and Object 5 appears and remains until t.sub.1 =30 seconds. The presentation terminates at t.sub.1 =30 seconds.
Now, we assume that the OMP shown in FIGS. 1B and 1C, as described above, is delivered by delivery system 10 of FIG. 1A which operates under the conventional client-server model. We also assume that the distribution network 30 operates at a network bit rate of 128 kb/s and that the memory capacity at the customer's CPE is 250 Kbytes. On receiving the customer's request for this presentation, a script of the presentation (the OMP and the Interaction Table) is transmitted from the server to the customer's CPE, and the CPE requests the objects of the presentation to be sent along the main presentation path of the presentation. In responding to the CPE's requests for delivery of the OMP, the server 22 begins by delivering objects along the main presentation path following Clock 1, with each object being transmitted as soon as requested. The CPE also tracks the size of objects to be transmitted to the CPE to ensure that the next object to be transmitted can be accommodated by the CPE memory. If the object cannot be accommodated, requests to the server are halted until the presentation time of previously delivered objects has passed, and objects are deleted from memory thus making memory available to store the next object.
Therefore, the CPE will request, and the server will deliver, Objects 1, 2, 3, 4, and 5, consecutively, with delivery start and end times as shown in FIG. 1E. Requests to the server are halted after Object 5 since the memory at the customer's CPE is full (250 Kbytes) and cannot accommodate additional objects at this time. Therefore, if the customer were to select the sidebar interactive feature after Object 5 has been sent to the CPE and before the interval ends at t.sub.1 =19 seconds, then the customer will experience a delay between the selection of the sidebar interactive feature and the presentation of Object 6. This delay would be attributable, in part, to the unavailable memory capacity at the CPE.
Even if the customer were to select the side bar at t.sub.1 =19 seconds, thereby essentially eliminating delays due to unavailable memory capacity, delays will also result due to interactive response time, which is the length of time between a customer's selection of an interactive feature and a change in the presentation reflecting the customer's selection. This response time includes the time necessary for the server to receive a request from the CPE reflecting the customer's selection, to react to the request, while simultaneously servicing requests of other customers, and to deliver objects to the CPE in response to the customer's interactive selection. Assuming that the customer's interactive selection is made at t.sub.1 =19 seconds and that the worst case interactive response time is 2 seconds, Objects 6 and 7 will be delivered with start and end times as shown in FIG. 1E.
The delays discussed above are largely a result of the reactive mode of operation of multimedia delivery systems operating under the conventional client-server model. Specifically, the server 22 delivers objects as requested by the customer's CPE along the main presentation path until the customer makes an interactive selection, and then the server reacts to the request of the customer's by transmitting those objects which fulfill the customer's selection as directed by the CPE. A drawback of such delivery systems is that the server in no way incorporates in its delivery strategy the delivery of objects that may be needed in the future to fulfill a customer's interactive selection such that delays or breaks in the presentation are eliminated.
Another major shortcoming of the server's reactive mode of operation is that the number of customers being served directly affects the interactive response seen by each customer. This means that as each new customer gains access to the system, the existing customers will likely notice decreased quality of performance, in the form of increased interactive response time. If one wishes to maintain interactive response time below a specified value, it is necessary to limit the number of customers or purchase a more powerful server. Otherwise, servicing more customers could result in significant contention at the server and ultimately interruptions or delays in the presentation of OMPs to customers. Since the server 22 must handle the delivery of OMPs to multiple customers simultaneously, it is important to properly scale the client-server ratio to ensure that delays in presenting the OMP will not exceed a preset maximum. This limitation imposed on the number of customers that can be serviced from a server affects the shared cost of the server among customers. Ideally, it is an objective to maximize the number of customers serviced by the server such that the cost of the server can be maximally shared among customers and widespread deployment of this delivery strategy can be achieved.
In view of the shortcoming of conventional approaches and mechanisms for delivering OMPs, an object of our invention is a system and method for providing interactive delivery of OMPs in real time. Another object of our invention is a delivery strategy which anticipates the object needs of the CPE along a main presentation path and delivers objects prior to the their presentation times. A further object of our invention is a delivery strategy which anticipates a customer's selections of an interactive feature and delivers objects that may be needed in the future to fulfill the customer's interactive selection such that delays or breaks in the presentation are virtually eliminated. Another objective of our delivery system and method is to increase the server-customer ratio realizable under a conventional client-server model without imposing unacceptable breaks or delays in the delivery of OMPs to customers.